The present invention relates to the composition and display of a three-dimensional image of a plurality of three-dimensional volume data obtained by an X-ray CT system, an MRI system, a 3D ultrasonic diagnosis system or an emission CT system, or more in particular to a three-dimensional image composition and display apparatus and a three-dimensional image composition method comprising a function of computing a two-dimensional projected image having a representative display surface depth (Z buffer) value in the same direction of the viewing line for each data, and a function of composing and displaying a translucent or opaque image of high image quality in correct three-dimensional position using a plurality of such two-dimensional projected images.
Well-known techniques relating to the present invention are disclosed in the following references:
(1) M. Levoy: Volume Rendering; Display of Surface from Volume Data, IEEE Computer Graphics & Applications, May 1988, vol. 8, No. 3, PP. 29-37
(2) Newell, M. E., Newell, R. G. and Sancha, T. L.: A New Approach to the Shaded Picture Problem, Proc. ACM. Nat. Conf., (1972) p. 443
This art is introduced in Nakamae et al, "3D Computer Graphics" Produced by Shokodo, P. 164.
(3) Visualization Machine, p. 12, by Mitsuo Ishii, published by Ohm
(4) High-Speed High-Performance Three-Dimensional System "Subaru", Vol. 4, High-Speed Plotting Mechanism, by Katsuhiko Nishikawa, Takahiro Sakuraniwa, Hideki Saito, Junichi Sugiyama and Akihiko Matsuo, p. 204, Collected Lectures 6 at Autumn Convention, 1992, The Institute of Electronic Information and Communication Engineers of Japan
(5) JP-A-1-37678 (originally published as JP-A-64-37678)
(6) Digitization and Three-Dimensional Image Processing of Medical Images, Asahi Chemical Information System Inc., Visual Information (M), May 1994, pp. 606-607
Reference (1) deals with the volume rendering of three-dimensional data. The three-dimensional data for volume rendering is considered to include translucent voxels. As a result of ray tracing from the view point toward an object, the opacity of each voxel is defined as the degree to which the light changes in transmittance as it passes through the translucent voxel, and the total sum of the light quantities reflected from the voxels is projected as a pixel value for a projection plane.
A simple method of composing a transparent or a translucent object by CG technique is Newell's one. This method is intended to express the transparency by mixing the color of a background object with that of a transparent object.
The technique of reference (3) uses the Z-buffer function as a method for processing a plane hidden by an overlapped object. In this technique, the surface position of each object model as viewed from the view point plane is compared with the value of the Z buffer. For the portion where an object is overlapped, the Z buffer near to the view point plane and the projection value of the particular object are rewritten, so that a projected image is obtained for all the objects by similar computations.
According to reference (4), a plurality of plotting mechanisms are connected through a depth data control mechanism so that images generated by the plotting mechanisms can be composed on the basis of the depth values. In this method, a plurality of images of a plurality of primitives defined in a three-dimensional space are generated in parallel by a plurality of plotting mechanisms. These images are composed to produce a three-dimensional image. In this method, therefore, the time can be saved by using a number of the plotting mechanisms.
With regard to references (2) to (4), in the case where a plurality of three-dimensional data are composed and displayed, it is necessary that portions to be displayed are three-dimensionally extracted from each data by being segmented, and segmented portions are embedded in the three-dimensional data to be integrated for the purpose of composition and display.
From the data integrated into a three-dimensional data this way, the technique of reference (5) extracts an arbitrary structure and produces the distance thereof from the projection plane. Surface images inside and outside a clipped region set in an arbitrary shape are composed and displayed in a frame of image.
Reference (6) discloses a 3D composition software called "Dr. View/Blender". This software permits display as viewed from a free direction. A portion of the display object is clipped, and an image of a different modality can be attached to the clipped portion in a predetermined ratio.
In taking a picture of an affected part by X-ray CT equipment, various angiographic operations are performed to obtain three-dimensional information on the network blood vessels and bone conditions at and in the vicinity of the affected part. Also, MRI can produce three-dimensional information on the condition of and the blood flow in the flesh with minimal invasive. Further, emission CT can generate three-dimensional information on the physiological functions of the human.
It is highly desired and required that the three-dimensional information thus obtained by various photographic methods are effectively utilized to contribute to the diagnosis and proposed operations, and data are mutually complemented to compose and display the data.
Conventionally, a volume rendering method for visualizing a display surface without uniquely determining it as in reference (1) is well known as a method for visualizing a single three-dimensional data such as described above with a high image quality. According to this method, which employs a visualization algorithm for causing several voxels of the smoothly-changing display surface to be involved in the projection value, it cannot be determined which voxel corresponds to the representative display surface depth (Z buffer) value. Also, the composition of a plurality of three-dimensional data requires a plurality of three-dimensional data to be displayed in a three-dimensional space and therefore consumes a great memory capacity. Further, for each tissue (region of interest) to be displayed translucently in such a manner that their stereoscopic overlapping conditions can be understood, an optimum parameter is very difficult to obtain due to the complicated setting of the rendering parameters.
Reference (5) discloses a technique in which each region of interest to be displayed is extracted from each three-dimensional data by segmentation. The segments thus extracted are embedded at corresponding positions of the three-dimensional data to be integrated for the purpose of composition and display. In this method, since segments associated with different data are embedded, the resulting image develops a discontinuous plane, which is expected to deteriorate the image quality at the time of composition and display by volume rendering.
Various image composition techniques including references (2), (3) and (4) are proposed for three-dimensional CG. These methods make it necessary that each region of interest of a plurality of three-dimensional data is modified into three-dimensional coordinate data such as the surface position data, expressed as a single three-dimensional vector data for rendering, and then rendered. For this reason, a volume rendering method with a superior image quality cannot be selected for each region of interest.
Furthermore, a composed image having a three-dimensionally conforming front-to-back order is impossible to produce by such a method as disclosed in reference (6) in which the result of a different data projection is simply attached to the projection result of three-dimensional data in a predetermined ratio to compose an image.